Fluctuation Theorem for Hidden Entropy Production 
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In the general process of eliminating dynamic variables in Markovian models, there exists a 
difference in the irreversible entropy production between the original and reduced dynamics. We 
call this difference the hidden entropy production, since it is an invisible quantity when only the 
reduced system's view is provided. We show that this hidden entropy production obeys a new 
integral fluctuation theorem for the generic case where all variables are time-reversal invariant, 
therefore supporting the intuition that entropy production should decrease by coarse graining. It is 
found, however, that in cases where the condition for our theorem does not hold, entropy production 
may also increase due to the reduction. The extended multibaker map is investigated as an example 
for this case. 

PACS numbers: 05.70.-a, 05.20.-y, 05.40.-a 
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Introduction. — Some of the main studies in statisti- 
cal physics concern reducing variables from the original 
microscopic equations to gain knowledge on the effec- 
tive dynamics. The reduction is typically justified by the 
time scale separation between variables, and the descrip- 
tions of effective kinetics become rigorous in the so-called 
Markovian limit 

0,1- 

Recent studies have focused on 
how physical quantities may vary according to the de- 
scriptions at different scales [3|, or are related to each 
other when defined in many partial systems 

In this paper, we consider how the irreversible (total) 
entropy production E varies by the general procedure of 
eliminating variables. Here, E is the sum of the Shannon 
entropy difference s and the heat entropy production a. 
This E, defined uniquely in a Markovian dynamics setup 
@, @|: has a positive ensemble average (denoted by (•)) 
value, 



(E) = (s) + (a) > 0. 



(1) 



Since the inequality (p} may be considered as the meso- 
scopic version of the second law of thermodynamics, the 
scale dependent descriptions of E is obviously an impor- 
tant matter. 

The main purpose of this paper is to clarify the be- 
havior of the difference between E defined in the original 
dynamics, and the entropy production E defined in the 
reduced system, 



E = E — E. 



(2) 



We call S the hidden entropy production. It is known in 
the context of infomation theory [7j that if E — ► E can 
be written as a reduction of variables in the Kullback- 
Leibler divergence by the Markov map, (H) is larger than 
zero. Corresponding to this notion, we first show for the 
case where all variables in the original dynamics are time- 
reversal invariant, that S obeys the integral fluctuation 
theorem, 



from which (S) > follows directly. 

An important situation where (H) > should not hold 
is when a time-reversal symmetric system (e.g. Hamil- 
tonian dynamics) is reduced to an irreversible stochas- 
tic dynamics. We find that Q does not hold if and 
only if the original dynamics includes time-reversal anti- 
symmetric variables and the symmetry of the density 
function is broken for them. As our second main re- 
sult, we demonstrate these points by investigating the 
extended multibaker model, which is a Hamiltonian-likc 
dynamics that reduces to a simple random walk in the 
mesoscopic regime. 

Model and definitions. — We consider the Markov chain 
dynamics on a continuous state space. The continuous 
variables x and y may each represent many variables, 
nevertheless we use a single variable notation. The time 
evolution of the probability density function P t (x,y) fol- 
lows the equation, 

P t+ At(x,y) = J dx'dy'P t (x',y')W x(t) (x,y\x',y'), (4) 

where At is the (infinitesimal) time step. 
W\(j){x, y\x' , y') is the transition probability from 
(x',y') to (x,y) between time t and t + At, and we 
assume that it is controlled by the time-dependent 
external parameter X(t). The integral by x' and y' in 
Eq. is taken over the whole space, and we have set 
j dxdyW\( t )(x,y\x' ,y') = 1. 

Let x N = (x ,xi, ...,x N ), y N = (yo,yi, — jJ/jv) be the 
stochastic path taken by (x, y) during the N time steps 
starting from t — 0. We define the stochastic Shannon 
entropy difference assigned to this path as, 



s(x N ,y N ) := log 



Po(xo,yo) 

probability 



(5) 



(e" H ) = 1, 



(3) 



Defining the path transition probability as 
W\(x N , y N \x ,y ) ~ Iljlo 1 W x{lAt) (x. l+1 ,y l+1 \xi,y l ), 
the path probability as P\(xN,y N ) := 
Po(xo,yo)W\(x N ,y N \x ,y ), and the Shan- 
non entropy of the whole system at time t as 



2 



S(t) := — f dxdyP t (x,y)\ogP t (x, y), we have 
(s(x N ,y N ))xM = S(NAt) - 5(0). Here the bracket 
(•)x,N denotes the average J dx N dy N P x (x N ,y N )-, 
where dx N := Y[^ =0 dxi and dy N := I^o^S/*- Using 
Pt{x) := J dyP t (x,y), we further define the coarse- 
grained stochastic Shannon entropy difference as, 



s(x N ) := log 



PNAt(XN) 



(0) 



The Boltzmann constant is set to 1 throughout the paper. 

Next we define the heat entropy production. The 
reverse trajectory of (xpf,y N ) is written as (x\f,y\f), 

where a;^ = (x N , xjf-i, x ), y^ N = (y N , Vn-i, —, Vo) 
with Xi being the time reversal of a;,-. Now the heat en- 
tropy production corresponding to the N step trajectory 
is, 



a{x N ,y N ) := log 



W\(x N ,y N \x ,y ) 



(7) 



Here, W X 1 is the transition probability assigned to 
the time-reversed protocol, W x \ {x* N , Vjf\%N, Vn) '■= 
]li=o w \{(N-i)At)( x i,yi\ x i+i,yi+i), which is defined us- 
ing A(i), the time reversal of X(t) [for example if the con- 
trol is by the magnetic field, X(t) corresponds to X(t) with 
reversed direction]. It is known that a(xN,y N ) corre- 
sponds to the entropy production induced by the energy 
transfer from the variables (x,y) to the hidden degrees 
of freedom in general stochastic models [!. 
Hamiltonian system including heat baths 
however that in particular Langevin models we introduce 
in the next section [(JTTJl and (f2Tj)], a(xN,y N ) does not 
directly correspond to the total heat dissipation Q. Let 
the coarse-grained path transition probabilities be, 



9] , and in the 
101. We note 



W x [x n \x ,Pq(-)]: 



dy 



P\(x N ,y N ) 



N~ 



Po{x ) 



w 



\t[x' N \x N ,P NAt 



(■)}■■= 



dy j 



AT! Vn 



N 



PNAt(XN) 



(8) 



(9) 



where 



P\t {x* Nl y^ Nl 

PNAt{xN,VN)W x i(x\,y\\x N ,y N ). Then the coarse- 
grained heat entropy production is defined as, 



a(x N ) := log 



W x [x N \x ,P (-)] 

W X f[x^ N \x N ,P NA t(-)} 



(10) 



Note that the coarse-grained transition probabilities 
W\,W\\ are in general non-Markovian [wc omitted the 
Po,PNAt dependence in the left hand side of Eq. (fT0|) ]. 

Let the total entropy production be S(ccjv,yjv) :== 
sIxn, y N ) + <j(xn, Vn)-> the coarse-grained total entropy 
production be T.(xm) '■= ^(xn) + ct(xn), and the differ- 
ence between them be, 



Z(x N ,y N ) '■= ^( x N,y N ) ~ ^(x N ) 



(11) 



This E(xn, j/jv) is the entropy production that could not 
be caught when only the x dynamics is observed, thus we 
call it the hidden entropy production. Now for S and £ 
the followings hold, 



>r s(x N ,y N )\ =1 

,-S(Xjv)\ = l 
I \,N 



(12) 
(13) 



Equation (|12p is the well known integral fluctuation theo- 
rem [(| [ll| . From this equality and the Jensen's inequal- 
ity we may show (E(x N , y N ))\ tN = (s(x N , y N ))\,N + 
(a(x^ 1 y N )) x ^ > 0, the second law corresponding to 
the whole Markovian dynamics. Eq. (|T3")) is the integral 
fluctuation theorem for the coarse-grained system. The 
special case of Eq. (|13p was mentioned in [12[ . 

Main theorem. — The first main result we present is 
that for the case where the variables x, y are time-reversal 
invariant (x = x, y = y), the following equality holds. 



-s(x N ,y N ) 



1. 



A, AT 



(14) 



This will be proved later. From Eq. (|14j) and Jensen's 
inequality, we see that the hidden entropy production is 
positive on average, 



(E{x N ,y N )), N > 



(15) 



which means that the entropy production decreases due 
to the elimination of y. 

As an example, we introduce a two-dimensional over- 
damped Langevin model Q. Taking 7 as the friction 
constant, T as the temperature, £(t) and £'(t) as the in- 
dependent Gaussian white noises with unit variance, the 
model is written as, 



ix = -k X (t)X - f\{ t )Oty + v / 2~/ T x{t) £(t) , 
IV = -~k x{t) a 2 y + f X (t)x/a + ^2 7 r A(t) £'(£). 



(16) 



Here, k > is the spring constant, / ^ is the am- 
plitude of the nonequilibrium force, and a > is the 
non-dimensional parameter that controls the time scale 
difference between x and y. In the a —> 00 limit, the 
fast-moving variable y could be eliminated from Eq. (|16[) 
by standard singular perturbation methods Q, and we 
obtain a closed dynamics of x, 

jx = -k m x+^T^t;(t). (17) 

We assumed that the modulation of k, /, T following the 
parameter X(t) is sufficiently slow compared to the time 
scale of the y dynamics. Note that in Eq. (fTTj) . the ap- 
parent nonequilibrium force due to / has vanished. De- 
noting the stochastic path from time to r by (x T , y T ), 
Eq. (fl"4"|) suggests that the hidden entropy production 3 
defined by Eq. (Ti~Tj) satisfies (exp[— S(sc T , y T )]) x T = 1 
for any a. In the particular case where a is sufficiently 
large, the coarse-grained transition probabilities ([5J and 
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© become equal to the Markovian transition probabil- 
ity corresponding to Eq. (|17|) . and the hidden entropy 
production may be written as, 



E(x T ,y T ) = lo. 



„ Pa{x ,yo)P T (x T ) 

3 P T (Xr,yr)Po(x Q ) 



(18) 



(It 



T 



X(t) 



-i ° f\(t)ay - y o k x{t) a y 



f\(t 



a 



is 



The second term in Eq. (Ql 
the difference between <j(x T ,y T ) = 

Jo t^T) i~ i ° ( kx + fm a v) + -y° ( k \(t) a2 y - /mWoO] 

and a(x T ) = — T dt [x o kx], where o represents the 

Stratonovich integral Q. If X(t) is fixed in time, we may 
show that S in the a — > oo limit has the steady average 
rate, 



(Z{x T ,y T )) X T /T I ^ fhk. 



(19) 



Since this rate is positive, we find that the reduced dy- 
namics written by Eq. (fTT)) does not reproduce the en- 
tropy production in the dynamics (fT6|) . See [1] for the 
general analysis. Note that Eq. (|14[) holds not only for 
the trivial case where S = 0, but also for (S) > situa- 
tions as in this example. 

The sufficient condition for Eq. (fl"4| to hold is that the 
original dynamics (before coarse-graining) only contains 
time-reversal invariant variables. Here we note on the 
case of eliminating the momentum variable (which is not 
time-reversal invariant) from the underdamped Langevin 
dynamics. We consider a one-dimensional model with 
position dependent temperature (j. 13|. 



P 



p/ m i 



F. 



\(t)(x) + y/2jT Kt) {x)Z{t). 



(20) 



with p the momentum variable, m the mass of the Brow- 
nian particle, F(x) the general x dependent force. Since 
p = —p, E(x T ,p T ) does not satisfy Eq. (IT4|) for general 
parameters. However, in the case where the parameters 
in Eq. (|20[) justify the elimination of p and therefore the 
effective dynamics obeys the overdamped equation, 



ix = F\(t)(x) 
we may prove 



dT m (x) 
dx 



+ v /2 7 T A(t) (x)o£W,(21) 



= 1. 



(22) 



This specific equality was noted in [13|. As in the previ- 
ous example, we find that (S) is positive for the general 
^ o case i G1E3 . 



dx 



Deterministic diffusion model. — Given the two exam- 
ples we have shown, one might expect that the decrease 
of entropy production is a general consequence of the 



Markovian limit coarse-graining. In this section we clar- 
ify that this is not the case by considering a determinis- 
tic Hamiltonian-like model. In this model, a probabilis- 
tic dynamics could be derived in the appropriate coarse- 
graining limit. Following the concept showed in the pre- 
vious section, we formally define the entropy productions, 
and check that Eqs. (fT4|) . (fT5j) are violated. We find that 
the violation is due to the asymmetry Pt(x, y) ^ Pt(x,y). 
This is the second main claim of this paper. 

Our Markov chain model is an extension of the multi- 
baker map [15j . The model is composed of many baker 
transformations that act on the nearest neighbor squares 
(Fig. [1]). The variables £,77 € [0,1] are the coordinates 
inside each unit area squares, and r 6 1,2, ...,L is the 
label of those squares. We set a periodic boundary con- 
dition for r, and regard r = L + las?*=l and r = as 
r = L. We further introduce the "discretized velocity" 
variable, v = + or — . The v — + and — systems are 
each composed of L squares, and are considered to be 
separated and non-interacting. The transition probabil- 
ity (deterministic map) from (r, £, r/, v) to (r', 77', v') in 
a unit time step is written as, 



'w+(r',e,v'\r,Z,v) (v = v' = +) 
W-(r',C,v'\r^,v) (v = v' = -) 
(otherwise) 



(23) 



with [see Fig. [T], 

W+(r', r/|r, £, rj) :=9{\- f) S^r-iS (£' - 2£) 6 (r/ - §) 
+6 (e - ±) 8 r ,, r+l 8 (£' -2^+1)8 (7/ - § - |) , (24) 

W- (/, rf\r, £, 77) ■= 6 (| - 77) 8 r> . r+1 8 (V - |) 8 (7/ - 2r?) 
+0 (77 - |) ^,^5 (C - f - ■§) * (77' - 27/ + 1) , (25) 

where #(•) is the Heaviside step function and 8(-) 
is the Dirac delta function. Taking the time rever- 
sal of the variables as (r,£,fj,v) = (r,£,r),—v), the 
model dynamics is completely time reversal symmetric, 
W(r', 77', v'\r, £, 77, v) = W(r, £, 77, -v\r', f, 77', -«'). 

Given an initial point (ro, £oj 7 ?cb u), we gain a deter- 
ministic trajectory after N time steps, {rN,£ N ,ri Nl v). 
Writing the probability density function of (r, £, 77, u) at 
time £ as P£(r, £, 77), the Shannon entropy difference after 
N time steps is zero, 

a (rjv, Cat, ti n ,v)= log — ^ = 0, 26) 

PNAt^N^N^N) 

because the phase space volume is conserved along the 
trajectories. We also find 



o-(r N ,£ N ,r) N ,v)=log 



W v (r N ,£ N ,ri N \ro,£o,r)o) 



W- v (r 1 N ,^ N ,rf N \r N ^ N , m ) 
0, (27) 
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FIG. 1. Scheme of the multibaker map. The map considers 
the label of squares (r), two-dimensional coordinates inside 
the square (£,??), and the discretized velocity (v). The map 
is area-preserving and time-reversal invariant since the tran- 
sition rules in the v = + and v = — systems are exactly 
opposite to each other [Eqs. (|23|l . (|25p ]. 



since we have set the dynamics in v = — to be 
the complete time reversal of v = +. Therefore we 
have confirmed £ (r N , £ N , r) N , v) = s (r N , £ N , r) N , v) + 
a ( r N, Cat i Vn-i v ) = 0- Note that this holds for any given 
initial density function Pq (to^OtVo)- 

Now we consider reducing the variables and 
only observing the discrete variable r. Then, the 
reduced total entropy production S(rjy) could 
be written using the "mesoscopic" probability 
distribution P t (r) := E.^/o'^fr.f.l). 
and transition probabilities Wt[r'\r,P^(-)] := 
E v I dZdTidfdrfPtir, e, r,)W v (r', rf\r, £, r?)/P t (r), 
^V|r,P± At (-)] :=£„ J dSdrrt'dr/P? (r,Z,r,) x 
W v {r'^',ri'\r 7 £_,r])/P^r). We find from Eq. (Bj! that 
for general initial distributions, the following holds. 



S (rw, £ N , = - ( S(rAr) ) < 0. (28) 



N 



To confirm that the (E(i"n)}n 7^ case exists, we assume 
that -Pq (r, £, rj) and P ~~(r, £, 77) are smooth in the £ and 
?7 direction, respectively, and define the time scale of the 
dynamics inside the squares as 



T£ tV := log sup 



(29) 



Then we may take, if L is sufficiently large, the "meso- 
scopic time scale" t* , satisfying T£ lV <f C r r . Here, r r 
(~ L 2 ) is the typical time for Pt(r) to become uniform. 
After this time t* , the dynamics reduces to a simple ran- 
dom walk in the r direction, 



Wt,W< 



t *>** 



> W(r'\r) 



— 5 r \r+l 



Is 

2 Or',r— 1 ■ 



(30) 



Retaking the initial time f = at this t* , the average total 



entropy production of the reduced dynamics satisfies 



''0) 

ro,ri,...,rjv 

P (r )W(r N \r 



x log ■ 



P N At(r N )W(r 



> 0. (31) 



Equality in (|3"TT) [and (|28p ] is achieved only when 

■Po(»"o)W(rjv|ro) = PNAt(r N )W(rlf\r N ) for all r/v, that 
is, only when the given initial distribution is the equi- 
librium state, Po(r) = l/L. Hence, we observe that the 
entropy production increases after the reduction in this 
model, as opposed to the case where Eq. (JT3J) holds. 

The inequality ([28]) states that the integral fluctua- 
tion theorem [Eq. (|14|)] does not hold in this model, ex- 
cept for the trivial case £ = (equilibrium state). From 
Eq. (j3"2"|) in the following section, we notice that the vio- 
lation of Eq. (fT4"|) is due to the broken symmetry in the 
density function, P^tO) PjvAt(')) which is valid for 
any N > including N — > 00 in this model. This is 
in clear contrast with the underdamped Langevin model, 
where the symmetry emerges in the overdamped (Marko- 
vian) limit [see Eq. (|34|)]. 

Proofs. — First we see that 



-s(x N ,y N ) 



A, AT 



dx N dy N P\(x N ,y N ) 



NAt 



(xn, VnWx^ (x\, y^ N \x N , y N )P (x )W\[x N \x } 

\ — := 7 

Po(xo,yo)W\(x N ,y N \x , yo)PNAt(x N )W x < l [x' N \x N ] 
= J dx N dy N P x ^(x t N ,y t N ) ^^-^ ' M ' 



P {xo)W\[x N \x } 

PNAt(x N )W X f [x\f\x N ] 



(32) 



For simplicity we omitted Po,P/vAt in W\,W\i, respec- 
tively. Now if x = x, y = y, the right hand side of 
Eq. (021) reduces to, 

t t \ Po(xoWx[x N \x ] 



/ dx N dy N P x t( 



V PNAt(x N )W^[x^ N \x N ] 

dx N P (x )W\[x N \xo\ = 1, (33) 



which is Eq. (|14j) . In the case of the underdamped model 
([20| . for a small parameter e := Tp/r^ (r x is the fastest 



time scale of the motion in the x direction, and t p := 
771/7), we may show that the ratio P T (x,p)/ P T (x, —p) 
becomes close to unity, 1 + 0(e), assuming that r is large 
enough compared to t p . Since e —y corresponds to the 
overdamped limit, Eq. (|32|) is now, 



P {x )Wx[x t \xq] 



P r (x T )W x t[xt\x T ] 



[1 + 0(e)] Xj (34) 
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which is Eq. ([Ml) . 

Remarks and conclusion. — Recently, entropy produc- 
tion from accessible degrees of freedom was experimen- 
tally measured and analysed in Although their 
definition of "apparent entropy production" is differ- 
ent to our coarse-grained entropy production, the heat 
[Eq. (fT0|) ] and the hidden entropy productions [Eq. (|TT|) ] 
are also measurable quatities in their experimental setup. 
Therefore, we claim that our main result Eq. (1141) can be 
experimentally tested. 

Next, we note the relation between our study and re- 
cent results on steady state thermodynamics. We ob- 
serve [1] that a(x) in the a —> oo limit of the model (fT^j) 
is equivalent to Hatano-Sasa's excessive entropy produc- 
tion In this scheme, S(x,y) is the so-called house- 
keeping entropy production, therefore Eq. ([M)) is equiv- 
alent to the integral fluctuation theorem shown in [18]. 
It is similar in the case of the model (PIU1) if (p?Tj) presents 
equilibrium dynamics Q , thus in this case Eq. (|22|) is the 
fluctuation theorem for the housekeeping heat extended 
to the underdamped Langevin model [19j. 

In conclusion, we have shown through various exam- 
ples that the entropy production may in general increase 
or decrease after reducing variables. For a generic class of 
systems with only time-reversal invariant variables, and 
for special cases where the final density is symmetric for 
all time-reversal anti-symmetric variables, Eq. (I14[) is sat- 
isfied and the entropy production always decreases. In 
the deterministic diffusion model, where the entropy pro- 
duction increases, Eq. (Q~4|) is violated due to the asym- 
metric density function. Since Eq. (|14j) has to be vio- 
lated for the total entropy production to increase after 
reducing variables, we find that such asymmetry of the 
distribution function is crucial in the general derivation 
of stochastic processes from Hamiltonian dynamics. It is 
left for future studies to clarify this point in more physical 



kinetic equations. 
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